Week 11: Data Analysis and Interpretation Part I

Dr. T. Kody Frey

Assistant Professor | School of Information Science

Housekeeping

Project Timeline

Only 6 weeks left! 1 group has been approved by IRB

  • Goal is to launch survey within the next 2 weeks
  • Final Qualtrics survey needs to be completed by 3/27. We will prepare them to launch during the workshop.
  • Data collection will last two weeks
  • Group should be working on front end (literature review, hypotheses, methods)

What Do You Know Now?

  • Research Design
    • Internal Validity
    • External Validity
    • Measurement Validity

You can learn a lot about a quantitative study using just these!

What Do We Still Need To Learn?

  • Analyzing Group differences
  • Analyzing Relationships

What’s Next?

Overview of Today

  • Research Design and Classification
  • t tests
  • Analyses of variance (ANOVA)
  • Complex tests of group differences
  • Discussion: Group Differences in Practice
  • Workshop: Midterm Review / Workday

GLM Ch. 18

General Design Classifications for Selection of Difference Statistical Methods

Key Terms

  • Each participant in the research is in one and only one condition or group
  • Participants receive only one level of the IV
  • DV only needs to be measured once
  • If power analysis says 20 in each group, then you need 60 total
  • Each participant receives or experiences ALL levels of the IV
  • If power analysis says 20 in each group, then you need 20 total
  • Appeal is in need for fewer participants acting as own controls
  • Impacted by carryover effects
  • Has at least one between-groups IV and one within-subjects IV
  • Has a minimum of two IVs

Understanding Design Types

It’s all about the levels of the IV

Designs are usually described in terms of:

  • the general type of design (between groups, within subjects, or mixed)
  • the number of independent variables
  • the number of levels within each independent variable.

What does this mean?

The researcher will typically state the design:

EX: A single factor design with 3 levels

EX: A 2 x 2 design with repeated measures

EX: A 3 x 3 x 2 factorial design

Single Factor Designs

Note: Factor is another name for IV

Single factor designs only have one IV

EX: The effect of time of class (morning, afternoon, night) on student engagement.

EX: Students’ perceptions of instructor effectiveness over time.

Class Example

Ikeola and Marian are examining campaigns using visual images can influence HPV vaccine behavioral intentions and eventual uptake of the HPV vaccine.

  • IV: Visual message (4 levels: threat, efficacy, threat x efficacy, control)
  • DV: Behavioral intentions

Between-Groups Factorial Designs

Includes more than one IV

EX: The effect of time of class (morning, afternoon, night) and student sex (male, female) on student engagement.

This is called a 3 x 2 factorial design.

  • The numbers represent the number of levels (3 levels of time of class and 2 levels of sex).

  • The number of numbers represents the number of IVs (time of class and sex)

Think Fast!

If you have a 4 x 2 x 3 factorial design…

How many IVs do you have?

What are the levels of each IV?

How many groups would you need for the study?

Class Example

Elizabeth, Madelyn, and Ansley are examining how message sender status on Instagram influences behavioral intention / adoption.

  • IV: Status (3 levels: influencer, layperson, control)
  • DV: Behavioral intention

This is currently a single-subject design. How would you make it a between-subjects design?

Within-Subjects Factorial Designs

Two or more IVs

Participants experience all conditions

EX: The effect of teaching style (traditional or inquiry based) on student engagement over time (pretest and posttest)

Example

To assess the CIS courses in the School of Information Science, we collect data from all course sections in August, December, January, and April to see if participation in course influences public speaking self-efficacy.

EX: The effect of class time on public speaking self-efficacy over time.

Mixed Designs

“Such a design might have two between-groups independent variables with three and four levels, respectively, and have one within-subjects independent variable with two levels.

It would be described as a 3 × 4 × 2 factorial design with repeated measures on the third factor.”

Basically, if it has a BG component and a WS component, it is a mixed design!

Summing it Up

If you recognize the design, you can generally determine the type of test needed.

GLM Ch. 19

The design classification ultimately drives the appropriate statistics.

Assumptions

  • Is it difference, associational, or descriptive?
  • What are the IVs and DVs?
  • How many IVs do you have?
  • What are the levels of the IV?
  • Is it a between, within, or mixed classification?
  • What is the level of measurement for each variable?

Three major statistical assumptions:

  • DV comes from population that is normally distributed
  • Variance of groups must be equal (homogeniety of variance)
  • Independence of observations

Choosing Your Test: Basic Differences

Choosing Your Test: Complex Differences

GLM Ch. 20

Data Analysis and Interpretation: Basic Difference Questions

Disclaimer

We are going to go through A LOT of tests.

I’m not going to show you how to run each one (i.e., what buttons to press)

IMO - it is more important to know which test is appropriate and how to interpret it. Once you figure out which test to run, there is a tutorial out there somewhere.

Chi-Square Test for Goodness of Fit

How does the proportion of smokers in our sample (observed) differ from the proportion of smokers (expected) in the population?

You need to know what to expect in the population (e.g., 20% are smokers; 80% are not)!

Chi-Square Test for Independence

A Chi-Square Test of Independence is used to determine whether or not there is a significant association between two categorical variables.

How does the proportion of smokers who are female and male in our sample differ from the proportion of smokers in the population who are female and male?

Example Data

t Test for Independent Samples

A statistical test conducted when there is an independent variable with two levels and a scale/continuous dependent variable.

Ex: Men will report higher levels of self esteem than women.

  • Check information about groups
  • Check assumptions
  • Assess differences between groups
  • Calculate effect size

Dependent (Paired Samples) t Test

A statistical test conducted when there is a within-subjects, independent variable with two levels and a scale/continuous dependent variable.

H: Fear of Statistics will be lower at time two than at time one.

  • Check information about cases
  • Check assumptions
  • Assess differences over time
  • Calculate effect size

Single-Factor (Oneway) ANOVA

1 IV, between groups, 2 or more levels

Normally-distributed DV

EX: Does total optimism differ by age groups?

*You COULD do this by running a whole bunch of t tests, but you inflate changes for Type I error.

Oneway Repeated Measures ANOVA

1 IV, within-subjects, 2 or more levels

Normally distributed DV

EX: Does confidence in coping change over time?

Complex Questions: Incorporating More than One IV

Factorial ANOVA

Performed in designs with more than one independent variable, two or more levels, and between groups design.

Normally distributed DV

EX: Do gender and age interact to influence reports of optimism?

Mixed between-within subjects ANOVA

Features one between-groups IV and one within-subjects IV

Normally distributed DV.

EX: Is there an interaction between condition and time on perceptions of fear of statistics?

Multivariate ANOVA (MANOVA)

One between-groups IV with 2 or more levels Two or more normally-distributed DVs

*Becomes repeated-measures MANOVA if you have within-subjects IV

EX: Do men and women differ in their reports of psychological wellbeing (3 variables: positive affect, negative affect, perceived stress)?

Your reading by Schrodt and Witt for today!

One-Way Analysis of Covariance

One between-groups IV with 2 or more levels

One normally-distributed DVs

One ore more theoretically-justified control variables

*Becomes repeated-measures ANCOVA if you have within-subjects IV

EX: Are there differences on fear of statistics for those who receive a math skills course versus those who receive a confidence building course while controlling for initial fear of statistics?

The effect of the intervention at T2 while controlling for values of the DV at T1.

Two-Way Analysis of Covariance

Two or more between-groups IV with 2 or more levels

One normally-distributed DVs

One or more theoretically-justified control variables

*Becomes 2 factor repeated-measures ANCOVA if you have within-subjects IV (or mixed design!)

EX: Do sex and type of course interact to influence fear of statistics while controlling for initial fear of statistics?

The effect of the intervention at T2 while controlling for values of the DV at T1.

Summary and Review

MOST designs are going to be simple; Complex does not mean BETTER

You now have a Fisher-Price stats toolkit :)

Rely on mentors, examples to guide you through more complex stats